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Management Response 



INTRODUCTION 

This management response (MR) serves two purposes: (1) to respond to the findings of the 
Youth Employment Strategy (YES) Eormative Evaluation by highlighting Human 
Resources and Skills Development Canada (HRS DC’s) accomplishments and (2) to outline 
an action plan for HRSDC to strengthen its role as lead department of this horizontal 
initiative. Additionally, the YES Eormative Evaluation and this response will form part of 
the information used for the design of the Summative Evaluation and further analysis 
required for the renewal of YES in 2008. 

KEY FINDINGS AND PROPOSED ACTIONS 

The report findings are organized under issues relating to governance, design and delivery, 
and performance measurement and success. 

Governance 

The formative report identifies several horizontal governance structures for the YES that 
are currently in place, including interdepartmental committee structures and coordination 
of agreements on information sharing. The report finds that there are elements of the 
governance structure that have yet to be implemented such as sharing of information, 
agreement on a common data collection tool and mechanisms for reporting on results. 

Eindings indicated that the authority of the lead department is largely undefined. In terms of 
the Interdepartmental Operations Committee, the report noted that there is little evidence 
of effective processes for coordinating activities, developing synergies or sharing best 
practices. In addition, there is no description of any decision-making processes for 
the Committee. 

The report suggests that the YES will benefit from clarification of the roles and 
responsibilities for the Interdepartmental Operations Committee and its general membership, 
including the lead department and the Treasury Board Secretariat (TBS). The report notes 
HRSDC’ s view that it should look for more direction from the TBS in obliging departments 
on key issues such as the use of the Social Insurance Number (SIN). 
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HRSDC is committed to building on its current work to further develop its 
leadership role. HRSDC also recognizes that the Government of Canada needs to create 
innovative and effective models for horizontal management initiatives to guide 
departments. As noted in the most recent Auditor General’s Report^ central agencies 
have a key role in monitoring horizontal initiatives and supporting lead departments and 
agencies. To build on our role as lead, HRSDC will benefit from increased guidance from 
central agencies on issues relating to managing and leading YES. 

Specifically, HRSDC has begun a dialogue with the TBS on how to strengthen its 
role as the lead department of the YES around issues of governance. In this context, 
there will be a review of YES accountability documents, particularly the Results- 
based Management and Accountability Frameworks (RMAF), in order to clarify 
roles, responsibilities and decision making authority for: 

• The lead department; 

• Interdepartmental committees and sub-committees; and 

• The TBS in relation to accountabilities of participating departments/agencies. 

To ensure that the Government of Canada is accountable for and transparent in its 
programming, HRSDC has been working with its federal partners to bring together 
all participant information collected from April I, 2003, to April I, 2006, for the 
YES summative evaluation. 

HRSDC is currently in the process of seeking authority to collect SIN related 
information for the YES, which will enhance overall evaluation activities and allow 
for program and policy realignments. Once HRSDC has been given authority, a 
second phase of the process will proceed for the other YES partners. 

Design and Delivery 

The formative evaluation found that the majority of departments have implemented YES 
programs in accordance with the umbrella terms and conditions. The report highlights 
that a particular strength of the YES is the wide range of work experiences available to 
youth participants through the variety of departments and agencies delivering programs. 

HRSDC will continue to ensure that programs are fully implemented as designed 
in accordance with the broader YES objectives and review YES accountability 
documents such as the RMAF in order to strengthen HRSDC’s role as 
lead department. 



' In the 2005 Report of the Auditor General of Canada to the House of Commons, a chapter was dedicated to Managing 
Horizontal Initiatives. The report underlines the fact that federal organizations need to work together effectively to 
achieve desired objectives hut not without guidance from Central Agencies such as the Treasury Board Secretariat. 
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Performance Measurement and Success 



The Formative Report found that data in year-end reports submitted to HRSDC were 
found to be incomplete for many participating departments. The report recognizes that 
management responsibilities for the YES are numerous and considerable. The findings 
state that management documents do not make references as to how these responsibilities 
would be carried out and that there is no budgeted line item to highlight costs and human 
resources required for this work. 

With existing resources, HRSDC has demonstrated strong leadership support by 
accomplishing the following data and performance measurement related activities: 

• In order for performance data to be collected and centralized, Memorandums of 
Understanding (MOUs) were developed specifying data and participant information 
sharing requirements between YES departments and HRSDC, with ten of twelve signed. 

• The MOU allows departments to forward participant information to HRSDC which 
was done through the development of a generic Participant Information Eorms (PIE) 
which have been shared with participating YES departments for implementation. 

• In terms of a common data collection tool, HRSDC has developed, implemented and 
will continue to maintain the YES Interdepartmental Data Collection System (DCS) 
which allows YES departments to transfer participant information to HRSDC. 
The DCS ensures that all participant information is gathered into a common database 
for evaluation purposes. All participant information as of April 1, 2005, will be entered 
in the system. 

• A joint letter from HRSDC and the TBS has been prepared for the YES federal 
partners to share information to support the YES Summative Evaluation for the 
three year period beginning in April 2003. 

Despite being the lead department, HRSDC does not have the authority to compel 
departments to share data, but is seen to be accountable for the YES as a horizontal 
initiative. HRSDC relies on other YES departments for participant information but 
does not have the necessary levers to ensure compliance. 

With the support of the TBS, HRSDC will ensure that participating YES departments 
be advised of the requirement to provide program information. This will allow HRSDC 
to exercise an appropriate leadership role in a horizontal initiative and develop 
approaches to ensure a complete and thorough data exchange between HRSDC and 
other federal partners. 

With respect to the horizontal measurement of results, it is the responsibility of YES 
departments to meet the minimum performance measurement expectations as outlined in 
the Common Performance Measurement Strategy in Section 4 of the RMAE that specifies 
the common performance indicators, data sources and collection frequency that are the 
responsibility of each participating department to provide. HRSDC is meeting the 
requirements of the Evaluation Strategy as outlined in Section 5 to conduct a high level 
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evaluation on the common objectives of all YES component programs through the 
development of the YES Summative Evaluation. 

Eindings of the report state that other departments delivering YES programs are not 
currently collecting data on all of the indicators outlined in the RMAE nor is there a 
process in place to transmit the data to HRSDC. No common data collection tool for YES 
programs has been implemented and this has resulted in inconsistencies in data collection 
across programs and in the frequency of collections. 

A review of YES documentation will also include departmental requirements to 
contribute to and adhere to horizontal reporting of results and the precedence of 
umbrella and departmental Terms & Conditions and guideline documents. 
The creation of sub-committees to include other departments in the process will be 
made available through the YES Interdepartmental Committee. 

FUTURE ACTIONS 

The formative report concluded there needs to be a clear definition on the scope of 
responsibilities for the lead department, and that HRSDC does not have sufficient levers 
or resources in place to exercise a leadership role. Considerable efforts and actions have 
been, and will continue to be, taken by HRSDC to develop strong leadership. In taking 
these actions, HRSDC will reinforce and expand its leadership role in terms of 
accountability for the YES. 

HRSDC is underscoring the importance of central agencies to provide guidance and 
support on issues related to managing YES as a horizontal initiative. While HRSDC is 
the lead on issues related to horizontal governance for the YES, the TBS is the authority 
on performance-measurement requirements for the Government of Canada. 

In addition to short term RMAE revisions, HRSDC is consulting with the TBS in 
terms of future YES programming, program renewal could include: 

• Requirements for YES departments and agencies to provide full and complete 
annual and mid-year reports to HRSDC on program outcomes and success 
measures in order to receive program funding; 

• Development of terms of reference for the governance structure; 

• Revised and clear definitions of roles and responsibilities for both the umbrella 
RMAE and department specific RMAFs clarifying the scope of responsibilities. 

The report highlights that YES management documents do not present a budgeted line 
item to highlight costs and human resources required to support work related to 
horizontal management. 

In this context, HRSDC will undertake a review of resources to ensure it can 
strengthen its horizontal management responsibility for the YES. 
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1. Introduction 



This report presents the findings from the 2004 formative evaluation of the Youth 
Employment Strategy (YES). It is based on the fieldwork and analysis that was conducted 
by Consulting and Audit Canada between September 2004 and December 2004. 

The report is organized in five sections: Section 1 provides background on the YES and 
on the evaluation objectives, scope and methodology. Sections 2, 3 and 4 present the 
evaluation findings in relation to governance, program design and delivery, and 
performance measurement and success, respectively. A summary of the main findings 
and conclusions are presented in Section 5. The evaluation report is supported by a series 
of supplementary technical documents (see Annex A). 



1.1 Background 

The YES was originally launched in 1997 with the objective of addressing some of the 
challenges faced by youth in making a successful transition from school to full-time 
employment. At that time, it offered funding for summer work opportunities for 
secondary and post- secondary students; and three internship programs for post- secondary 
graduates in international, and science and technology fields; and Eirst Nations 
on-reserve and Inuit. It also included two other government of Canada initiatives: Youth 
Internship Canada and Youth Service Canada which were designed to create entry-level 
and other work opportunities for youth. 

In 2002, following evaluations of several components of the YES, the review of new 
research on youth employment, and consultations with stakeholders, the Strategy 
underwent a redesign. The primary objective of the realigned YES is to assist Canadian 
youth in enhancing their employability skills, while increasing the number of .skilled 
young Canadians in the workforce. 

The redesigned YES is a $395 million interdepartmental initiative. It involves 14 federal 
departments and agencies and 31 of their programs.^ As such, it is described and designed 
to operate as a horizontal initiative that leverages the efforts of multiple federal government 
organizations to achieve the YES objectives. Eigure 1 presents a program overview of the 
realigned YES, including the participating departments and agencies, and the planned annual 
allocation of funds. As Eigure 1 depicts, the YES comprises three program streams: 

• Career Eocus: activities delivered under this program stream provide post-secondary 
graduates with career-related work experiences and learning opportunities, mentoring 
and coaching that will build advanced skills and encourage the continuation of 
advanced studies. 



^ During the drafting of this report, it was announced that the Department of Fisheries and Oceans’ Science and 
Technology Youth Internships Program would end in fiscal 2004-05. 
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• Skills Link: the focus of this program stream is to help youth overcome barriers to 
labour market participation through tailored interventions. These barriers could include, 
but are not limited to, challenges faced by recent immigrant youth, youth with disabilities, 
lone parent youth, youth who have not completed high school. Aboriginal youth, and those 
youth living in rural or remote areas. 

• Summer Work Experience: this stream targets secondary and post-secondary school 
students who are intending to return to school full-time, with short-term opportunities 
to build career-related skills through work experiences. 



Figure 1 

The Youth Employment Strategy® 



$265 M from HRSDC A-base budget C 



Youth Employment Strategy 
HRSDC (lead department) 

^ ! $395M I , 




H $130 M from Treasury Board for all programs 



3 Program Streams 
14 Departments 
31 Departmental Programs 



Career Focus 




Skills Link 




Summer Work Exoerience 


$59M 


^$216M^ 


^$120M ^ 


11 Departments 
17 Departmental Programs * 




3 Departments 
4 Departmental Programs * 




5 Departments 
10 Departmental Programs * 



Foreign Affairs Canada (1) 
Environment Canada (2) 

National Research Council (1) 

Natural Resources Canada (1) 
Canadian Food Inspection Agency (1) 
Fisheries and Oceans Canada {1 ) 
Agriculture and Agri-Food Canada (1) 
Industry Canada (3) 

Canadian Heritage (2) 

Canadian International Development 
Agency (1) 

Human Resources and Skills 
Development Canada (3) 



Canada Mortgage and Housing 
Corporation (1) 

Indian and Northern Affairs (2) 
Human Resources and Skills 
Development Canada (1 ) 



* Number in brackets indicates number of sub-programs 



Canadian Heritage (3) 

Parks Canada (1) 

Industry Canada (3) 

Indian and Northern Affairs (2) 
Human Resources and Skills 
Development Canada (1) 




^ The three boxes at the bottom of the diagram offer estimates of cash funds leveraged by the departmental programs. 
Not all programs require that funds be leveraged. The estimates are based on the annual reports prepared by 
departmental programs and may not account for all funds leveraged. In addition, these estimates do not include in-kind 
contributions; the YES does not have criteria or guidelines for counting such contributions and estimates provided by 
departmental programs may not have been comprehensive or comparable. 
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1.2 Evaluation Objectives and Scope 

This study was a formative evaluation of the horizontal aspects of the YES. This statement 
makes two very important points. First, as a formative assessment, it was designed to 
identify any YES management, design or delivery issues that had emerged during this 
implementation phase. Secondly, to the extent possible, the study was to determine if there 
were any early indications of program benefits. Note that this study did not assess the 
departmental programs as individual entities, but rather, how they operate collectively to 
achieve the overall goals of the Strategy. 

The evaluation was guided by an Evaluation Working Group. The group was comprised 
of representatives of four YES departments - Human Resources and Skills Development 
Canada (HRSDC), Industry Canada, Canadian Heritage, and Indian and Northern Affairs 
(which collectively account for just over 90 % of allocated funds) the Treasury Board 
Secretariat (TBS), and evaluation consultants from Consulting and Audit Canada. 

An Evaluation Steering Committee, consisting of representatives from all 14 participating 
departments and TBS, met to make required decisions based on recommendations from 
the Evaluation Working Group. Resulting from extensive review and discussion by the 
Evaluation Working Group, and approval by the Evaluation Steering Committee, 
seven specific questions were examined in the evaluation, falling under the general issues 
of governance, design and delivery, and performance measurement and success: 

• Is there an effective governance/ accountability structure for the YES as a whole 
(i.e. horizontal management)? 

• Is the governance/accountability structure within participating departments effective in 
relation to the operation and delivery of YES? 

• Are the YES departmental initiatives, as designed and implemented, consistent with the 
realigned YES policy goals, program objectives and operational requirements? 

• How is YES differentiated from other employment programs/initiatives that serve 
youth in the 14 participating departments? What is the extent of complementarity with 
these other programs? 

• What approaches to delivery have been taken by participating departments? What are 
the strengths and weaknesses of the different approaches? What improvements could 
be made? 

• Are program monitoring and reporting activities sufficient to support effective program 
accountability and decision making? Is sufficient data being gathered to support the 
summative evaluation? 

• What have been the direct, short-term benefits to youth who have participated in YES? 
To employers? 
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1.3 Evaluation Methodology 

The evaluation was eondueted in two main phases: a planning phase, undertaken between 
April and August 2004, at whieh time the evaluation questions, seope and researeh plan 
were finalized. The related data gathering aetivities and assoeiated analyses for the 
evaluation were undertaken between September 2004 and Deeember 2004. The evaluation 
employed four key methodologies: doeument review, interviews, a literature review and an 
expert workshop.'* 

Document Review: As indicated above, there are 14 departments and 31 departmental 
programs that are involved in the YES. Each department has its own program 
documentation. As such, CAC compiled and reviewed the extensive array of umbrella 
and departmental YES documentation. Key umbrella documents reviewed included: the 
YES Realignment Program Policy Design Paper; the Treasury Board Submission, 
including supporting annexes; the Interdepartmental Operational Guidelines; the draft 
Preliminary Privacy Impact Assessment Report (PPIA), including the interdepartmental 
Memorandum of Understanding (MOU) to be signed between HRSDC and participating 
departments; past evaluations of YES; and the Umbrella Results-Based Management and 
Accountability Eramework (RMAE) and Risk-Based Audit Eramework (RBAE). 
Program- specific documents included: departmental program Terms and Conditions 
(Ts&Cs) for YES; contribution agreements; RMAEs; data collection tools; program 
brochures; and previous studies. In total, approximately 400 documents were reviewed in 
support of the evaluation. 

Interviews: Interviews comprised a key component of the evaluation research. A total of 
198 stakeholders were interviewed, and included: program officers/managers in the 
National Capital Region and in five regions; senior managers; third-party delivery agents; 
and Treasury Board representatives. 

Literature Review: A literature review was undertaken to identify, review and analyze 
the most recent research, theories and models, in Canada and abroad, related to: youth 
transitions; youth employment; human development/skills development; and related 
evaluation approaches, indicators and methodologies. 

Expert Panel: In November 2004, an expert workshop was held to provide an 
opportunity for YES program managers and evaluators to learn from experts about the 
challenges faced in designing, implementing and measuring the results of youth 
employment and skills development programs. 



It was decided during the planning phase that it would he impractical to survey youth and employers in the formative 
evaluation, given the available time and resources, and the lack of access to personal contact information for 
participants. The supplementary technical documentation provides additional detail on the methodology, including a list 
of interviewees, documents reviewed, and the findings from the literature review and the expert panel. 
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2. Governance Findings 



2.1 Horizontal Governance 

Q. 1 Is there an effective governance/ accountability structure for the Youth 
Employment Strategy (YES) as a whole (i.e. horizontal management)? 



Context 

Horizontal governmental initiatives present special governance challenges because they 
are generally designed to be collaborative in nature. On the one hand, they seek to 
maintain an individual department’s independence and accountability for programs and 
program decisions under its control. On the other hand, they seek an enhanced value 
through interdepartmental cooperation and coordination. Typically therefore, they adopt 
governance models that stress cooperation, consensus and information-sharing; and they 
avoid vesting significant power in one participant, although one entity will generally 
assume a lead role. 

As would be the case for horizontal initiatives in general, the dual obligations of governance 
for YES remain: (1) to provide vision and direction; and (2) to ensure effective oversight. 
The challenge is achieving these requirements within a collaborative environment. 

The governance design of YES places considerable emphasis on interdepartmental 
information-sharing, collaboration and coordination, while clearly identifying HRSDC as the 
lead department. Three documents serve as sources for the design of YES governance. 

• The umbrella Results-based Management and Accountability Eramework (RMAE), 
which is authoritative since it has been approved by Treasury Board Secretariat (TBS); 

• The Interdepartmental Operational Guidelines of March 26, 2003, which have not been 
accepted by participating departments, but which nevertheless reveal the intention of 
the design and the governance structures for YES; and 

• The draft (Interdepartmental) Memorandum of Understanding (MOU) that was intended 
for signature between HRSDC as lead department and each participating department to 
specify the delivery and management obligations and the data/information sharing 
requirements at the client level for each department. Although the MOU, l ik e the 
Guidelines, has not yet been signed, it serves as a source document for the intention of 
the design and the governance stmcture of the realigned YES initiative. 

Horizontal governance of YES is to be achieved in two ways: 

• Through the program approval and related ongoing management processes of TBS and 
departments, in which program results and terms and conditions are negotiated and 
defined, and departmental authorities are approved at regular intervals - in this case, 
five years; and 
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• Through an interdepartmental committee structure that is focused on planning, information 
sharing across departments, and collaborative priority- setting and decision-making. 

Regarding departmental requirements, in addition to the umbrella RMAF, individual 
departments, as required, prepared department-specific RMAFs for their YES programs 
which were also approved by TBS. In some cases, these provide additional terms and 
conditions. This is reflected in the Umbrella RMAF as follows: 

• “...all participating departments will be accountable for the design, management, 
implementation and results achieved through their respective YES initiatives, within the 
umbrella Terms and Conditions that apply to the Youth Employment Strategy.” 

• “Participating departments will continue to be responsible to their respective Ministers for 
the results achieved by their initiatives under the Youth Employment Strategy.” 

Thus, the overall vision of YES is embedded in its design at the approval stage; and 
oversight provisions for individual YES programs are built into those TBS approvals, 
both through the normal requirements of managing and reporting on any government 
program, and through special provisions such as terms and conditions in both umbrella 
and department- specific RMAEs. The major decisions respecting their desired results, 
delivery strategies and overall priorities are made at relatively long intervals of time, 
when their authorities come up for renewal. The realigned YES programs, implemented 
in April 2003, are approved to March 2008. 

The interdepartmental committee structure is clearly designed to be operational in nature. 
The RMAE identifies the principal committee as the YES Interdepartmental Operations 
Committee. It has two specialized sub-committees reporting to it: a Communications 
Committee, to develop a communications strategy and “take responsibility for coordinating 
and disseminating all communications material related to YES;” and an Evaluation Steering 
Committee, to lead the evaluation of YES “in its entirety,” drawing on the lead department’s 
evaluation group. 

The Guidelines identify the Interdepartmental Operations Committee as “the primary 
consultative body for the fourteen departments and agencies,” and is “responsible for matters 
relating to planning, operational policy, program delivery, reporting and evaluation, and 
communications.” It further details a series of specific roles for the committee, including: 

• Planning and priority-setting (setting common goals and priorities); 

• Coordinating the in-year reallocation process; 

• Reviewing progress in achieving common goals; 

• Making decisions on program issues that require consensus; and 

• Other specific management and information sharing functions, including the sharing of 
best practices related to program delivery. 
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The Guidelines further lay out a specific planning process for the Interdepartmental 
Operations Committee, identifying a minimum of three meetings per year, with: 

• A January meeting to set priorities for the year; 

• A September meeting to review progress and reallocate funds; and 

• A November meeting to prepare for the January priority-setting meeting. 

According to the Guidelines, all departments have “two interconnected roles,” with: (a) a 
“principal task” of delivering their own YES initiatives; and (b) a second role of membership 
in the interdepartmental “community.” This is further defined in the draft MOU: 

• Through interdepartmental coordination, both Parties will undertake, collectively with 
other YES participating departments and agencies, planning and priority-setting 
activities, conduct mid-year reviews and review progress at agreed-upon times during 
each fiscal year. 

• The Parties will review, on an annual basis, the full menu of YES activities to identify 
gaps and program priorities with a view to respond to emerging issues, maximize 
efficiency and avoid duplication. They will also draw on the results of interdepartmental 
planning to direct their own specific planning activities. 

The role of HRSDC as lead department is generally described differently in different sections 
of these documents. Eor example, the RMAE states that “As HR[S]DC is ultimately 
accountable for attaining the expected results for the Youth Employment Strategy, HR[S]DC 
will have the ultimate decision-making authority for issues related to the overall policy, 
design and implementation of the YES.” In particular, “the Youth Initiatives Directorate, 
Employment Programs Branch, National Headquarters, Human Resources Development 
Canada is accountable for the overall delivery of the Youth Employment Strategy.” It is to 
achieve this accountability through chairing the Interdepartmental Operations Committee, 
and through MOUs with each participating department. 

The Guidelines document identifies Human Resources and Skills Development Canada as: 

“. . .the lead department responsible for the overall coordination of the YES. It is 
responsible for coordinating a consistent delivery approach across the 
Government of Canada. HR[S]DC’s role includes coordinating the delivery 
model for the overall program and policy management, results monitoring, 
evaluation activities and financial accountability.” 

The RMAE describes this somewhat differently: 

“As lead department, HR[S]DC chairs and is responsible for co-ordination and 
management of an inter-departmental committee structure that will oversee the 
implementation of the Youth Employment Strategy.” 
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In summary, the governance design for YES as a horizontal initiative embeds the longer- 
term vision of programs in their approved design; requires participating departments to 
oversee performance through normal management mechanisms; and creates a horizontal 
governance structure and processes for the ongoing, operational planning, priority-setting 
and in-year adjustment of YES as a whole. Participating departments are expected to 
focus mainly on their own programs, but to contribute to the horizontal aspects of the 
initiative as well. The lead department is “accountable” for YES results as a whole, 
while playing a coordinating role and managing an interdepartmental committee structure 
to oversee the implementation of the Youth Employment Strategy. 

This design gives full play to the independence and integrity of departmental programs. 
It is heavily dependent on: (a) good will and collaborative relationships among 
participants; (b) the facilitative leadership skills of the lead department; and (c) reliable 
performance information provided by departments and analyzed by the Interdepartmental 
Operations Committee. 

There may be an implied provision for conflict resolution in the RMAE’s statement that 
HRSDC has ultimate decision-making authority for “overall policy, design and 
implementation.” However, there is no definition of decision-making procedures, processes 
or authorities and the matters on which decisions need to be made remain general and 
somewhat theoretical. The authority of the lead department is undefined. In practice, its 
authority (its control) appears limited to: (a) its contribution to the development and signing 
of MOUs with participating departments; (b) its role in collecting, analyzing and reporting 
program performance information; and (c) its own resources and programs, which might be 
expected to give it considerable weight as a source of knowledge, expertise, and practical 
assistance. The three design documents make no provision for a secretariat to support the 
governance structure. 

Findings 

Implementation of the governance structures - committees and meetings 

The Interdepartmental Operations Committee is in place. The Committee has met 
formally since the implementation of the realigned YES in April 2003. Bilateral 
meetings have also taken place between the lead department and other members to deal 
with specific results measurement issues as appropriate (see next paragraph). Members 
have also engaged in informal, short discussions on operational issues in the context of 
the Evaluation Steering Committee and the Evaluation Working Group meetings. 

Eour formal meetings of the Interdepartmental Operations Committee have been held 
over the period following the realignment of YES. The meetings took place in 
May 2003, September 2003 March 2004, and November 2004. These four meetings did 
not address all activities outlined in the planning cycle, such as coordinating plans, 
assessing results and making course corrections as intended. These meetings were mainly 
focused on the problem that has challenged the program since 2003: the issue of 

collecting and transmitting participants’ social insurance numbers (SIN), along with other 
personal socio-demographic and contact information, as a means of tracking youth for 
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purposes of program evaluation. They were also preoccupied with the related issue of 
the MOUs between the lead department and participating departments and agencies, 
as the draft MOU assumes departmental agreement to collect and transmit the 
participants’ individual information to HRSDC. A number of other formal and informal 
meetings have taken place in attempts to resolve these problems. 

Departments were invited to raise issues or bring information to these Interdepartmental 
Operations Committee meetings. There is little evidence of effective processes for sharing 
information, coordinating activities, developing synergies, or sharing best practices on 
program delivery. Meetings have been large, and the levels of participants have varied 
considerably, from director to officer level. Some departments routinely send several 
participants. Senior managers generally delegate attendance. Many departments complain of 
short notice for these meetings. 

A key responsibility for this Committee is the annual reallocation of program funds from 
departments who will not be investing all of their allocation, to other departments who 
require more funds. This reallocation process did take place in both 2003-04 and 
2004-05, as planned. 

The design intention of the Communications Sub-committee is to develop communication 
strategies and products for YES as a whole. The Committee met formally in October 2003 to 
discuss the secure website and how to gain access to data maintained in it. At this meeting, 
they also discussed possible upcoming meetings of the committee and the Communications 
workplan for 2003-04. Members of the committee also met on two other occasions to share 
information about their department specific programs. In the fall of 2003, they met to share 
information about their programs/initiatives, targets, and best practices. During the winter of 
2004, they met to share information on their programs and initiatives. These information 
days were welcomed by departmental staff. 

The Evaluation Steering Committee has been active in overseeing this formative 
evaluation and has created a working group that has also been active. The creation of 
such a sub-committee is consistent with the governance design, which leaves room for 
such structural adjustments. The Committee met about once a month between April 2004 
and September 2004. The Working Group met approximately every week during the 
same period. 

Horizontal Decisions 

As noted in the preceding subsection on Context, the design of YES governance does not 
describe any decision-making processes, or define the nature of decisions the 
Interdepartmental Operations Committee would be expected to take. Nor does it define 
the nature of the authorities the Committee would exercise, or anticipate a need for a 
conflict resolution approach or mechanism. It posits the collaborative sharing of 
planning and performance information, with mutual agreement to adapt departmental 
program plans where desired. 
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At the time of YES realignment, it was recommended that action be taken to improve the 
measurement of relevance and success of the YES in order to inform renewal discussions 
in 2008. Experts in the field of performance measurement and evaluation of employment 
programs highlight the importance of client information. The YES has not yet developed 
the detailed information collection practices and tools necessary for the horizontal 
tracking and reporting of the outcomes of youth who have participated in the program. 
The findings related to this subject are addressed in two separate sections of this report. 

• Those related to the kind of performance information that is essential for management 
discussions of YES program implementation and the monitoring of participant / program 
outcomes as part of routine performance management activities; and the kind of 
information required for periodic evaluation of program impacts, future policy discussion, 
and policy and program development are reported in section 4 below; and 

• Those related to the exchange of information between federal departments for purposes 
of horizontal analysis and reporting are addressed immediately below. 

In order to exchange personal information on youth participants from the participating 
department to HRSDC, departments must formally sign Memoranda of Understanding 
(MOUs) which specify the data elements to be exchanged and the processes that will 
govern the protection and use of this information. In the case of YES, each department is 
to collect and transfer specific data elements to the lead department. The lead department 
is to create a centralized database for purposes of horizontal management, accountability 
and evaluation. Oversight for this activity rests with the Interdepartmental Operations 
Committee. HRSDC, as the lead department, has been responsible for initiating and 
managing the drafting of related documents and for engaging departments in discussions 
that are to ultimately lead to the signing of the MOUs. 

The lead department and the YES Operations Committee, with the advice of TBS, 
generally have devoted a considerable amount of time discussing the issues related to 
collection of client information, including the MOU and the collection of the SIN 
(see above). Transmittal of the SIN and other client information to HRSDC would 
provide some variables related to the employment history of YES participants; this could 
contribute to effective evaluation of achievement towards ultimate program outcomes. 

The collection and use of the SIN, like all personal information, is carefully controlled in 
the federal government environment. The use of the SIN requires specific TBS authorization 
and is seen as an important element of client information. Because of its accuracy in 
identifying clients in administrative records, the SIN facilitates the tracking of participants 
across programs and over time. It also offers efficiencies in the collation of factual 
information from existing administrative databases on annual income (including earnings and 
Employment Insurance receipt), layoffs, family dependents, etc. Access to this data generally 
improves the quality and accuracy of performance data overall, offers more information for 
ongoing analysis of results, reduces the burden on program participants to respond to 
surveys, and helps to control costs of obtaining information for reporting on program results. 
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With the transmittal of the SIN a key element in the MOUs, many departments viewed 
this as a significant change in the collection of personal information. They had not 
required their program participants to provide this piece of information in the past. 
They had not used connections to existing administrative databases to estimate the 
possible effects of their programs. Some were concerned that requiring participants to 
provide this piece of information as a condition of participation would adversely affect 
their outreach and the decisions of youth to participate. 

The result is that almost two years into the realigned program, these MOUs have not been 
signed and the performance data that has been collected has not been centralized. As noted 
above, the MOUs were intended to be one of HRSDC’s and the YES Interdepartmental 
Operations Committee’s key tools for effectively coordinating YES as a whole and 
responding to its accountability for overall YES results. 

Another horizontal information-gathering and reporting issue that has eluded YES 
decision-makers is the lack of a common architecture for data collection. Eor example, 
departments do not have common tools for program exit questionnaires or follow-up 
questionnaires. This leads to data and information inconsistency across the YES as a whole 
- a significant problem, particularly in an initiative so reliant on reporting at the horizontal 
level as a means of ongoing management and results assessment (see Section 4 below). 

Relationships for Collaborative Horizontal Management 

Several interview respondents characterized meetings of the Interdepartmental 
Operations Committee as adversarial. Different perceptions of key roles may account for 
some of the strain evident in some relationships. 

• HRSDC takes a limited view of its role as lead department. It sees itself as only a 
coordinator, with no authority over other departments. Eurther, it asserts that it is not 
resourced to play a significant role in the horizontal governance of YES, beyond calling 
essential meetings and attempting to resolve contentious issues such as the collection of the 
SIN. In its view, it pulled the RMAE together voluntarily, with great difficulty 
(an achievement about which several departments commented favourably). It sees its 
current responsibility as primarily to its own programs. It looks for more direction from 
TBS in obliging departments to cooperate on key issues, such as the SIN. 

• Departments value their own YES programs highly and are primarily focused on their 
own mandates over the broader YES objectives. While they have faithfully attended 
horizontal meetings and submit reports to the lead department, most do not take 
ownership of horizontal problems, such as the lack of an agreed MOU, or the lack of 
viable solutions to data collection issues. 

• TBS sees a role for itself at the design and approval stage of such a horizontal initiative, 
and a monitoring and advisory role throughout the initiative’s life-cycle. Eor example, 
TBS provides advice in the interpretation of TB policies and other issues, but does not see 
grounds for taking unilateral corrective action where program integrity is not in question. 
Based on the RMAE and requests for funds, TBS believes that HRSDC is adequately 
resourced and has the tools it needs to exercise its governance responsibilities for YES as 
a whole. 
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Ongoing oversight 

In its governance design, YES posits an oversight role for the Interdepartmental 
Operations Committee through ongoing information sharing, planning and adjustment. It 
also provides for a robust evaluation capacity through: 

• Creation and approval of the umbrella RMAF and Risk-Based Audit Framework 
(RBAF), with departmental alignment as a condition of participation; 

• The Evaluation Steering Committee, and the requirement for both a formative and 
omnibus summative evaluation; and 

• Requirement for annual YES reports, both by departments and by HRSDC for YES as 
a whole, that are intended as instruments of effective oversight. 

However, effective oversight depends on the decision-making capacity and authority of 
the oversight body or bodies - in this case, a combination of the Interdepartmental 
Operations Committee, the separate management regimes of the participating 
departments, and Treasury Board ministers when they consider possible program renewal 
in the future. 

The Interdepartmental Operations Committee has not exercised such oversight so far, 
and there is no evidence that departments have adopted horizontal findings in their own 
program oversight. An important question for YES governance will be how it will 
make decisions to adjust the program in response to the findings of the current 
formative evaluation. 



2.2 Internal Governance 

Q. 2 Is the governance/accountability structure within participating 
departments effective in relation to the operation and delivery of YES? 



Context 

Each participating YES department or agency is accountable for the management of its 
own YES program or programs. This is a fundamental principle in the design of YES. 
However, the YES design requires that the outcomes of participating departmental 
programs be aligned with defined YES outcomes. The YES design does not impose any 
internal governance models or requirements on any participating department or agency. 
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The level of complexity associated with governance requirements for YES within a 
department will relate to the number of YES programs in the department and their alignment 
to the department’s own program outcomes. In departments with more than one YES 
program, there may be a need to coordinate across YES programs, to ensure consistency in 
communications, program tools and program access; to speak with a single departmental 
voice at horizontal YES meetings or on YES issues; and to champion YES internally. 

Findings 

Nine of the 14 participating departments and agencies have only one YES program and thus 
face relatively simple internal governance requirements. In these cases, YES programs are 
managed in ways consistent with each department’s existing accountability structure. 

Of the remaining five departments, two have taken a formal and coordinated approach to 
internal governance. Indian and Northern Affairs Canada has placed all four of its YES 
initiatives within a single directorate to achieve management synergies and ensure 
coordination across YES initiatives. Canadian Heritage has established a secretariat to 
ensure internal coherence and consistency, develop common tools, and to achieve 
internal management efficiencies. In both cases, this enables a single departmental 
representative to speak with a single voice at horizontal YES meetings or on YES issues, 
and to champion YES internally. 

HRSDC, Industry Canada, and Environment Canada meet internal governance needs by 
relying on informal contacts between specific YES programs, existing departmental 
committees and forums, and the horizontal YES governance structure external to 
each department. 
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3. Design and Delivery Findings 



3.1 Implementation 

Q. 3 Are the Youth Employment Strategy (YES) departmental initiatives, 
as designed and implemented, consistent with the realigned YES 
policy goals, program objectives and operational requirements? 

Context 

To assess whether the YES has been implemented as designed, it is important to identify 
the overall goals and specific objectives of the Initiative. While the YES has focused 
on assisting youth to prepare for, obtain and maintain employment since its inception, 
the realigned Strategy introduced or stressed some new key principles: 

• an emphasis on skills development^ as the key to positive labour market outcomes; 

• a much stronger focus on youth facing barriers to employment; and 

• the adoption of a client-centred approach in the delivery of the YES. 

These broad policy goals are reflected in the design of the program and related 
documentation, such as the Youth Employment Strategy Realignment: Program Policy 
Design Paper, the YES Ts&Cs, Umbrella Results-based Management and Accountability 
Eramework (RMAE) and Risk-based Audit Eramework (RBAE), the draft Memorandum 
of Understanding Human Resources and Skills Development Canada (HRSDC) and 
participating departments/agencies, and the Interdepartmental Operational Guidelines. It 
was this documentation that comprised the basis for the assessment of implementation. 
These documents were reviewed to determine the specific expectations of the realigned 
Strategy. Departmental programs (through documents and interviews) were then 
reviewed to assess the extent to which they align with these expectations. 

This analysis was complicated by three factors: 

• The YES umbrella documentation is inconsistent with respect to descriptions and 
definitions of program goals, objectives and activities; 

• Not all the documents identified above have been agreed to by all departments and 
agencies; and 

• In addition to the umbrella documents, participating departments were required to 
submit a mini-RMAE and departmental Terms & Conditions (Ts&Cs) to Treasury 
Board Secretariat (TBS) for approval. In most cases, these departmental documents 



^ Skills generally refers to employability skills, which are defined very broadly (based on the work of The Conference 
Board of Canada, 1999) to include a range of academic, personal management and work-related skills. 
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were virtually identical to the umbrella documents; in others, however, they specify 
particular ways in which the departmental YES programs may deviate from the 
umbrella requirements. 

These three factors made it very difficult to conclusively define the YES program design 
and, correspondingly, to assess implementation. The evaluation addressed these internal 
variations by distinguishing between program requirements (the parameters found in 
documents that have been agreed to by all participating departments, such as the 
Umbrella Ts&Cs, the RMAEs and/or RBAEs), and guidelines (program features specified 
in documents, primarily the Interdepartmental Operational Guidelines, that are still in 
draft form). Where departmental Ts&Cs exempt a program from a particular design 
parameter, it is noted. 

Findings 

Career Focus ( CF) 

The objective of CE programs is to contribute to the supply of highly qualified people in 
Canada by supporting the acquisition of advanced employability skills and facilitating 
school-to-work transitions. This is normally achieved through the provision of career- 
related work experiences in Canada and abroad. The specific requirements and guidelines 
for the 17 Career Eocus programs are presented in Table 1 below, together with the general 
findings related to the various parameters. 

Most CE departmental programs are meeting all or most of the program requirements, 
with the exception of the client skills assessment. This point is important, as the client 
skills assessment is intended to support the new YES emphases on skills development 
and client-centred services. There was only one of the 17 departmental CE programs that 
can be said to have implemented the client skills assessment process as intended: that is, 
it administers a detailed pre- and post-intervention skills assessment, and requires 
participants to develop a related action plan to address their developmental needs. 



Table 1 

Career Focus Departmental Programs: Summary of Findings in Relation to 
Program Requirements and Guidelines 


Current Program Requirements 


Findings for Departmental Programs 


Youth must be aged 15-30 


• Participants faii within the specified age range (and tend to be at 
the upper end of the range). 

• Maximum age of 30 is seen as restrictive. 


Participants shouid be 
post-secondary graduates 


• CF program participants are generaiiy, but not always, post- 
secondary graduates; in some cases they may still be in school. 
Departmental Ts&Cs for four YES programs recognize that 
participants may be students. 


Participants are unempioyed or 
underempioyed 


• CF programs meet this criteria. 


Ciient needs/skills assessment must 
be done 


• Although half the CF programs have some evidence of a skills 
assessment tool, they are very basic, pre-assessments only, 
with no action plan, and frequently not mandatory, or not 
administered to all participants. 

• Only one program has implemented a skills assessment process 
in keeping with that specified in YES documentation. 
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Table 1 (continued) 

Career Focus Departmental Programs: Summary of Findings in Relation to 
Program Requirements and Guidelines 


Current Program Requirements 


Findings for Departmental Programs 


Programs are expected to leverage 
funds 


• All but one departmental program has some type of leveraging 
requirement. 

• One departmental program hires youth internally (no external 
partners). 

• The amount leveraged in 2003/04 was $1 1 .8M in cash and 
$30M in-kind. Looking at cash contributions only, this 
represents 20% of the total CF budget. 


Multi-year contribution agreements 
(up to three years) are allowed 


• Most CF programs (9/53%) use one-year agreements. 

• Those departments that allow three-year agreements often do 
not use them. 


Interdepartmental Operational 
Guidelines 


Findings for Departmental Programs 


Youth should be out of school 


• Five of the programs may have participants who are still in 
school. In four cases, this is recognized in the departmental 
Ts&Cs and/or mini-RMAFs. 


Work experience should relate to field 
of study 


• This is generally the case. 


Youth can participate only once 


• All departmental programs restrict participation to one work 
experience in that program. 


Work experience must be no longer 
than 12 months 


• Duration of work experiences, with one exception, is limited to 
12 months or less. 


Maximum cost/participant is $15,000 


• All programs, but one, have a maximum cost/participant that is 
$15,000 or less. 

• Interviewees frequently commented that the maximum cost has 
not increased for many years and is no longer sufficient. 


Work experiences should be 
supported by mentoring/coaching 


• A large number of CF programs (7 of 17) have a formal 
requirement for mentoring. 



Although they are not currently formal requirements, most CF programs are also meeting 
most of the operational guidelines: for example, work experiences generally relate to the 
participant’s field of study; all programs restrict participation to one work experience 
(in that program); and almost half of the departmental programs (7 of 17, or 41%) have a 
formal requirement for mentoring or coaching of youth participants. 

Looking at both program requirements and guidelines, there are two CF departmental 
programs that vary noticeably from the overall model: 

• Canadian Food Inspection Agency’s Youth Employment program operates as an 
internal recruitment program, providing youth with two-year internships, to which YES 
contributes $20,000 per year. The goal is that these interns will be hired by the Agency 
at the end of the internship. 

• HRSDC’s International Academic Mobility (lAM) program offers graduate students 
the opportunity to study an international curriculum either in Canada or abroad. 
The YES Terms & Conditions recognize I AM as an anomalous program within 
the Strategy. 
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Skills Link (SL) 



The objective of the SL program stream is to help youth facing barriers to employment to 
develop a broad range of skills and knowledge in order to participate in the current and future 
labour market. Unlike CF, which is delivered primarily through work placements, 
SL programs support a menu of support services, skills development and work experiences. 

The four SL programs are quite diverse in the approach they take to assist youth: two are 
much like CF internships, where youth are hired for specific work experiences for up to 
one year; one is designed to provide youth with career-related information and assistance, 
primarily through co-op programs and career fairs; and one offers youth a continuum of 
supports, ranging from individual skills enhancement to group-based employability .skills 
workshops, to actual work placements. 

The specific requirements and guidelines, and related finding for Skills Link programs 
are presented in Table 2 below. 

While SL departmental programs appear to be addressing the general objective of assisting 
those who face significant barriers to employment to develop knowledge and employability 
skills, there is substantial variability in the extent to which they adhere to the specific 
requirements and guidelines outlined in the YES documentation. As designed, the YES SE 
program emphasizes the importance of client assessment, tailored interventions to address 
specific skill requirements, and case management of participants. This approach has been 
adopted by only one of the four departmental programs, albeit the largest, accounting for 
93% of allocated funds. 



Table 2 

Skills Link Departmental Programs: Summary of Findings in Relation to Program 

Requirements and Guidelines 


Current Program Requirements 


Findings re Departmental Programs 


Youth must be aged 15-30 


• Participants In all but one program fall within the specified age 
range. The one exception is authorized through the 
department-specific terms and conditions. 

• Tend to be at the lower end of the range. 


Participants should be out of school 


• SL program participants are, with one exception, out of school. 
Again, this exception has been allowed In the department’s 
Ts&Cs. 


Participants must be in need of 
assistance to overcome 
employment barriers 


• SL programs select based on such criteria. 

• In three of the four SL programs, clientele are First Nations 
youth. 


Client needs/skllls 
assessment required 


• This is a program requirement for one of the SL programs. 

• In the other three programs, client needs assessments may be 
done at some sites. 


Programs are expected to 
leverage funds 


• Only one of the SL programs requires some contributions 
(from private sector employers). 

• The program stream leveraged $3.4M In cash and $0.09M In- 
klnd contributions is 2003-04 (1 .6% of the total SL budget). 

• Almost all the SL contributions were generated by one of the 
three participating departments. 
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Table 2 (continued) 

Skills Link Departmental Programs: Summary of Findings in Relation to Program 

Requirements and Guidelines 


Current Program Requirements 


Findings re Departmental Programs 


Multi-year contribution agreements 
(up to three years) are allowed 


• Two of the three departments with SL programs use one-year 
agreements. 

• Use of multi-year agreements by the third department varies 
from one region and office to another. 


Interdepartmental Operational 
Guidelines 


Findings re Departmental Programs 


Youth may participate In more than 
one Intervention 


• Although youth can take part In more than one SL intervention, 
It Is discouraged or not applicable In two of the four programs. 

• One SL program Is designed to “move” youth through a 
continuum of interventions, gradually building their 
employability skills. 


Case management is required for 
youth accessing employment services 


• One of the four departmental programs has a formal 
requirement for case management. 



Summer Work Experience (SWE) 

The SWE program is intended to help secondary and post- secondary students acquire 
employability and/or career-related skills and support them in financing and furthering 
their education. These objectives are normally achieved by working with other 
government departments, non-government organizations, the private sector, and other 
orders of government to create short-term employment opportunities. Table 3 outlines 
the specific requirements for the ten SWE programs. The documentation for SWE 
programs did not specify additional design requirements. 

There were no real changes to the SWE program in the 2003 realignment. As Table 3 
indicates, overall, the SWE programs are meeting most of the requirements, although 
there are one or more exceptions in relation to each of the four parameters. 

There are two SWE programs that differ notably from the general program model: 

• Parks Canada uses YES to hire internal summer staff. This appears to be supported by 
the Umbrella Terms & Conditions (in which participants are included among eligible 
program recipients) and their departmental RMAE; 

• One of the two Indian and Northern Affairs Canada SWE programs supports learning 
events (for example, science and technology camps), rather than work experiences. 
Their Departmental Ts&Cs address this difference in approach and also specify that 
youth younger than 15 may participate in this program. 
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Table 3 

Summer Work Experience Departmental Programs: Summary of Findings 
in Relation to Program Requirements 


Current Program Requirements 


Findings re Departmental Programs 


Youth must be aged 15-30 


• Majority of programs stipulate age requirement. 

• Programs serving Aboriginal youth (four) sometimes have 
participants outside the age range. In one case, this is 
recognized in the departmental Ts&Cs. 


Participants should be secondary or 
post-secondary students planning to 
return to school 


• All but two programs specify that students should be returning 
to school. 

• No penalty for not doing so. 


Programs are expected to 
leverage funds 


• More than half the SWE programs (6/60%) leverage cash and 
In-kind contributions; four programs have no related 
requirement. 

• One departmental program hires youth internally (no external 
partners). 

• The total amount leveraged In 2003/04: $20. 6M in cash $9.2M 
in-kind; looking at cash contributions only, this represents 17% 
of the total SWE budget. 


Contribution agreements are a 
maximum of one year 


• All departmental programs, except one, restrict agreements to 
one year; the exception Involves a delivery agent that manages 
related CF and SWE programs under one three-year 
agreement. 



3.2 Overlap / Duplication 

Q. 4 How is YES differentiated from other employment programs/initiatives 
that serve youth in the 14 participating departments? What is the extent 
of complementarity with these other programs? 

Context 

As a horizontal initiative, it was important to assess the extent to which YES is sufficiently 
inclusive, and to identify any potential overlap, duplication or complementarity with other 
non- YES programs. A cursory review of federal youth employment programs indicated that 
a comprehensive environmental scan would require more time and resources than were 
available for the formative evaluation. It was therefore decided to limit the scope of this 
assessment to other youth employment programs within the 14 participating departments. 
The objective was to determine the features that distinguish YES from other programs and, 
in so doing, to lay the groundwork for a more comprehensive assessment of federal youth 
employment programs in the summative evaluation. 

Other employment programs that serve youth were identified by: members of the 
Evaluation Working Group; a review of programs listed in the Government of Canada’s 
publication. Youth Link; and an Internet search. Thirty-one non- YES programs were 
identified in the 14 participating departments. These programs were reviewed in relation 
to key YES parameters (objectives, eligibility, operations, duration and financial 
information) to determine if they were sufficiently similar to the YES to warrant further 
investigation. Some additional follow-up interviews were conducted with YES program 
managers and managers of non- YES programs. 



Formative Evaluation of the Youth Employment Strategy 




Findings 



The review of non- YES programs indicated that there are quite a number of federal and 
departmental programs that are very similar, in design and operation, to YES. 

• One program in EedNor operates exactly like a CE program. It provides post-secondary 
graduates in Northern Ontario with work placements, with the goal of leading to longer 
term employment. The major difference is that the wage subsidy for the EedNor 
program is substantially higher than that allowed in CE ($27,500, compared to $15,000). 

• The Canadian International Development Agency has a number of youth programs 
that, like YES, provide international work experiences (for a similar length of time), 
but that do not have employment as a long-term goal of the program. 

• Within HRSDC there are several programs with goals and interventions very similar to 
those in YES, but that can serve all working age individuals, not just youth. 

• Two government-wide programs, the Eederal Student Work Experience Program 
(ESWEP) and the Eederal Public Sector Youth Initiative Program (EPS YIP), are very 
much like CP and SWE respectively, except that the work opportunities provided are 
within the federal government. However, as was noted in the previous section, there 
are two YES programs that also hire youth to work in their agencies. 

• There are also several small programs in National Research Council and Natural 
Resources Canada that provide youth internships internally. 

There appear to be no formal linkages, at the design and operational levels, for 
communication or coordination between YES and these similar non- YES programs, 
although this may be taking place at the departmental or program level. 

Given that the research identified a variety of non- YES programs with objectives, 
clientele and/or activities similar to those in the realigned YES, it was concluded that the 
Strategy is not clearly distinguished from other youth employment programs. 
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3.3 Delivery Models 

Q. 5 What approaches to delivery have been taken hy participating 
departments? What are the strengths and weaknesses of the different 
approaches? What improvements could he made? 

Context 

With respect to delivery, the evaluation examined the two different management approaches 
(i.e., centralized vs. decentralized) and three different delivery models (i.e., direct to youth, 
direct to employer and third-party) being used by departmental programs. 

Under a centralized approach, the program is administered by headquarters and regions 
are generally not involved in delivery. With a decentralized approach, regional offices 
play a key role in program delivery, while the overall responsibility for program 
administration remains with headquarters. From a departmental program perspective, 
14 of the 17 Career Focus programs and 7 of the 10 Summer Work Experience programs 
are managed centrally. However, from a budget allocation perspective, for SWE, HRSDC 
is allocated $90 million of the $120 million and it is using a decentralized approach to 
deliver its Summer Career Placements programming. All SE programs use a decentralized 
delivery model. 

Departments are using one (or a combination) of three different delivery models: direct to 
youth, direct to employer, or third-party. Departments using direct to youth delivery 
work directly with the youth and provide the work experience. A contribution agreement 
is signed between the department and the youth. When the YES program is delivered to 
the employer, the department works with an employer that provides the work experience 
to the youth. A contribution agreement is signed between the YES department and the 
employer, who may, in turn, sign a contract or agreement with the youth participants. 
In the third-party delivery model, the YES department works with an organization 
(generally referred to as a delivery agent or community coordinator) that is responsible 
for delivering the program as outlined in the contribution agreement. The third party 
signs an agreement with employ er(s) to provide the work experiences for the youth. 
There are four departmental programs that use direct to youth delivery, 14 that use direct 
to employer, and 20 that use third-party delivery. 

Findings 

Interviewees were asked to identify the various strengths and weakness of these 
approaches and models, as well as possible improvements and viable alternatives. 
Because there was no comparative data on results and delivery costs of the departmental 
programs, there was no basis for recommending one model or approach over another. 
Tables 4 and 5 provide a summary of these benefits and challenges. 
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Table 4 

Benefits and Challenges of Centralized and Decentralized Approaches 




Centralized 


Decentralized 


Benefits 


• Program has a single, 
national identity 

• Delivery is more consistent 

• Have critical mass necessary 
to deliver program 

• Program gets more attention at 
Headquarters 


• Program officers know communities; can tailor 
interventions, respond to issues more effectively 
and efficiently, monitor more easily 

• Allows for more flexibility/ autonomy 


Challenges 


• Hard to be responsive to 
all regions 

• Only delivery agents with 
national coverage can 
participate 

• More difficult to monitor 


• Consistency of delivery can be an issue 

• Requires substantial communication between 
regions and Headquarters 

• Tends to be more expensive, although costs are 
often “absorbed” by distributing across whole 
delivery network 



Table 5 

Benefits and Challenges of Delivery Models: Direct to Employer and Third Party 




Direct to Employer 


Third Party 


Benefits 


• Department has more control 
over program; can monitor 
projects more closely; and 
have more confidence in 
program delivery. 

• Builds/supports department's 
local networks/partnerships 
(i.e., greater visibility). 


• Maximizes benefits of existing networks. 

• Delivery agents have expertise in area. 

• Reduces workload of department (no capacity to 
deliver). 

• Delivery agents can be and/or are perceived to be 
more responsive. 

• Delivery agents develop/strengthen their 
own networks. 


Challenges 


• Increases workload for 
departmental staff. 


• Department not as close to delivery and therefore 
programs are harder to monitor and ensure 
delivery is being done as intended. 

• More partners (i.e., departments, delivery agent, 
employer) means more goals that have 

to be addressed. 


• Administrative burden (for employers, delivery agents, departments) is very heavy. 
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4. Performance Measurement 
and Success Findings 



4.1 Performance Measurement 

Q. 6 Are program monitoring and reporting activities sufficient to support 
effective program accountability and decision making? Is sufficient 
data being gathered to support the summative evaluation? 

Context 

To assess this question, four areas were examined: data collection; the co-ordination of 
data collection and reporting; use of data by departments for decision-making and 
reporting; and the sufficiency of data for the summative evaluation. 

The requirements for data collection for the horizontal initiative are outlined in the Youth 
Employment Strategy (YES) Umbrella Results-based Management and Accountability 
Eramework (RMAE). A common performance measurement strategy, outlining specific 
program outcomes, indicators and collection methods was developed for each of the three 
YES program streams. The RMAE indicates that “departments will be responsible for 
monitoring and reporting on their respective initiatives at a minimum level outlined in 
this RMAE.” 

The YES Umbrella RMAE and the draft Preliminary Privacy Impact Assessment (PPIA) 
outline how data collection and reporting was to be coordinated at the horizontal level. 
The Interdepartmental Operations Committee, on a recommendation from the YES 
Interdepartmental Evaluation Committee was to provide a common survey regime for all 
YES programs, that would include pre- and post- intervention, follow-up and employer 
exit surveys. 

They were also to develop a Participant Information Eorms (PIEs) to be used to collect 
participant and outcome data immediately post-program. The draft PPIA indicated that 
PIEs would be completed by each participant and the participating YES department 
would forward completed forms to HRSDC, where they would be entered into a common 
database. Work on the electronic capture of PIEs was underway by the end of the 
reference period for this evaluation - with some departments already engaged in 
the functioning of the electronic system. The common reporting mechanisms at the 
horizontal level were to be mid and year-end reports. 
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Findings 

Data Collection 

Table 6 provides information on the number of departmental programs that are collecting 
performance information. The number in parentheses in the first column represents the 
number of YES departmental programs in a particular YES program stream. 

Most YES programs are currently using some kind of tool(s) to collect basic participation 
and/or program benefits data from participants. About two-thirds of the departmental 
programs collect data from participants (21 programs) and employers (20 programs) 
immediately post-program, using surveys or questionnaires. Eew departments (four) 
conduct any follow-up surveys with youth^. Only one program conducts any follow-up 
with employers. There are a total of ten programs that are not collecting information 
from youth at program exit. Consequently, tracking of programs’ results is mostly 
limited to data collected at program entry, exit or shortly after intervention. 



Table 6 

Number of Programs Using Tools to Collect Outcome Data 


YES Stream 
(Total YES Programs) 


At Program 
Exit (youth) 


At Program Exit 
(employers) 


Follow-up 

(youth) 


Follow-up 

(employers) 


Career Focus (17) 


14 


11 


3 


0 


Skills Link (4) 


1 


3 


0 


0 


Summer Work 
Experience (10) 


6 


6 


1 


1 


Total (31) 


21 


20 


4 


1 



A review of data collection tools against the immediate and intermediate outcome 
indicators in the RMAE found that the programs are not collecting all of the data as per 
the strategy. Eludings for each of the three YES program streams are presented below. 

Career Focus (ten immediate and intermediate outcome indicators in RMAE) 

Sixteen of the 17 CE programs (94%) collect data on the number of youth completing 
work experiences. Many (14 programs / 88%) also collect profile data for participants 
and youth views on enhanced employability skills (13 programs). Just over half collect 
data on youth satisfaction with the work experience, employer satisfaction, and on the 
number of youth in work experiences related to their expertise or field of study. 
Two programs (12%) collect data on employers’ views with respect to enhanced youth 
employability skills. 

Although most of the departmental programs collect data on whether participants are 
employed (76%) or went back to school (88%) at program exit or shortly after, a much 
smaller number ask whether alumni are finding career-related employment 
(six programs) or are returning to advanced studies (no programs). Given that Career 



^ One CF program contacts youth 30 and 90 days after participation; one CF program does follow-up after 6, 12 and 
24 months; and the third CF program recently did a survey of youth who had participated in their program in the past 
five years. There is also one SWE program that does follow-up a few months after participation. 
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Focus is designed to help graduate students find career-related work, these latter 
indicators appear to be more relevant. 

Skills Link (13 immediate and intermediate outcome indicators in RMAF) 

All four departmental programs track the number of youth participating in interventions, 
and three (75%) track the number of program alumni finding employment and the 
number of youth that return to school at program exit or shortly after. Data on whether 
youth are enhancing their employability skills is fairly limited. Only one of the 
four programs asks youth and employers their views on whether employability skills have 
been enhanced. 

Summer Work Experience (ten immediate and intermediate outcome indicators in RMAF) 

Seven of the ten programs (70%) collect information on the profile of youth participants. 
Just under two-thirds (six programs) collect data on: youth working in experiences related 
to expertise or field of study; dollar amounts paid to youth; number of weeks worked; 
youth views on enhanced employability skills; and number of youth returning to 
post- secondary education. 

Coordination of Data Collection and Reporting 

No common data collection tools for YES programs have been implemented. This has 
resulted in inconsistency in data collection across programs, including how questionnaires 
have been designed (i.e. specific questions to be included, how questions are asked^) and 
frequency of administration. Often questionnaires are completed in hard copy (few programs 
have electronic databases) and are stored in various locations depending on the program 
delivery structure (e.g., with regions, delivery agents, or at Headquarters). 

Although a draft Participant Information Form (PIF) was provided by HRSDC, not all 
programs have implemented the form. Seven programs have adopted the form; 16 are 
using a form similar to the PIF and are meeting most of the data collection requirements 
as per the draft interdepartmental MOU. Eight programs are not meeting the participant 
data requirements in the draft MOU. Even where the PIE has been implemented, the hard 
copies are not yet being forwarded to HRSDC because, without a signed 
interdepartmental MOU, the department is not able to accept them. 

As was intended, the mid and year-end reports are the mechanisms currently used to 
report on YES at the horizontal level. A template for the reports was developed by 
HRSDC. There are five categories of reporting: funding (including leveraged funding), 
number of participants, employment equity numbers, geographic location of participants, 
and results (i.e., the number of participants who found employment or returned to school) 
at program exit or shortly after. 



’ For example, with respect to employability skills gained, some programs ask youth for a narrative description of skills 
gained, where others provide a checklist and ask participants to identify which skills were increased. This variation in 
how the questions were asked did not impact on documenting whether a departmental program collected information 
or not. 
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A review of the year-end reports identified some concerns with the reporting process. 
Five of the 31 programs (16%) did not submit a year-end report for 2003-2004. Of the 
programs that did submit a report, only one had complete numbers reported in all 
five categories; ten reported numbers in all categories, but the numbers did not always 
account for the entire population of participants; and 14 reported some numbers, but not 

o 

in all categories. Reporting issues seem to stem from the fact that data systems do not 
facilitate reporting of such information, or that delivery agents or regions fail to provide 
complete data when reporting to national staff. 

HRSDC has previously developed annual reports for YES (a formal one for fiscal 2001-02; 
and a status update report for fiscal 2002-03). A horizontal report for fiscal 2003-2004 has 
not yet been drafted. 

Use of Data 

Most programs use participant data to compile the year-end reports that are to be shared with 
HRSDC. However, according to some interviewees, there is a lack of time and/or resources 
to do any further data analysis. Views with respect to the utility of the data varied: 
in approximately half the departmental programs, one or more interviewees indicated that the 
data is used to make program changes or improvements, although few specific examples 
were provided. Generally, this observation was made by delivery agents for the Career 
Focus or Skills L ink program streams. Conversely, one or more stakeholders in 75% of the 
departmental programs indicated that the data is not used to make program decisions. 
In cases where interviewees claimed that data was not being used, several also expressed a 
growing disincentive to continue collecting the information. 

Sufficiency of Data for Evaluation 

As discussed above, many departmental YES programs are not currently collecting data on 
all of the indicators outlined in the RMAF; nor is there a process in place to transmit 
the data to HRSDC. This will present problems when it comes to assessing the impact 
of the YES in the summative evaluation. In addition, however, the discussion of 
implementation {Section 3.1) alludes to the fact that the RMAF indicators may not capture 
the variety of strategies/ interventions currently used by departmental programs to improve 
employment outcomes. Without a clear articulation of the program theory and expected 
outcomes and gains for participants (i.e., whether the programs seek to improve youth 
employment through skills development, the provision of career-related information, 
job experience, or a combination of these or other factors) the summative evaluation will 
not be able to understand or attribute any identified results to the programs themselves. 
In addition, the summative evaluation has not yet been designed in detail. This adds to the 
difficulty of assessing the sufficiency of data in relation to assessing program impacts over 
the longer-term. 



One program (SchoolNet First Nations) is not accounted for because for the 2003-2004 program year the data was 
reported with that for SchoolNet. 
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4.2 Success 



Q. 7 What have been the direct, short-term benefits to youth who have 
participated in YES? To employers? 

Context 

Three key sources of data were used to examine YES program benefits: the year-end 
reports; interviews; and program documentation provided by program representatives. 
The year-end reports and program documentation were the main sources of quantitative 
data. As discussed previously, however, the data in the year-end reports is not complete 
for many participating departments. In addition, existing program data on results was not 
validated. Interviews and program documentation provided qualitative information. 

Findings 

Benefits to Youth 

A review of the 2003-2004 departmental year-end reports revealed that most programs’ 
reports provided information on the number of participants. However, only ten of the 
17 CF programs, none of the SL programs, and three of 10 SWE programs provided 
some results information.^ The information reported for each program stream were: 

• Of the 3,480 youth reported as participating in Career Focus internship programs in 
2003-2004, the status of 745 participants at program exit is known: 542 were employed 
(not necessarily in their fields of study); and 203 went back to school (level of studies 
not specified). 

• A total of 26,866 youth participated in Skills Fink in 2003-2004. 

• Of the 79,232 youth who participated in the Summer Work Experience internship 
programs in 2003-2004, the status of 964 participants at program exit is known: 897 
went back to school; and 67 were employed. 

Interviewees said that evidence of program benefits for CF and SWE can be found in 
post-program questionnaires and surveys. However, very few programs actually compile 
or analyze this data. The program managers for five of the 17 CF programs and four of 
ten SWE programs provided additional program documentation, for 2003-2004, that 
contained additional quantitative evidence that CF programs and SWE are achieving 
some benefits. For example: 

• 80% of interns in one CF program were employed; 90% of those in their field of study. 

• 75% of interns in another CF program were employed by initial employer/sponsor. 



® These numbers obtained from the year-end reports vary from those provided in the 2003-2004 Horizontal Initiative 
Template provided by HRSDC. However, this template does not provide sufficient detail to explain the variances. 
The figures for HRSDC’ s Skills Link program were provided through interviewees. 
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• 67% of participants a SWE program thought the summer job would help them get 
full-time work in their chosen field. Of those, 59% indicated that the on-the-job 
experience and skills development (34%) were the main reasons why. 

For SL, two of the four programs provided some results information, however, 
the information was not national data, but rather, was provided by one delivery agent. 

Qualitative information on program benefits was gathered both through interviews and 
program documentation. Most interviewees were very positive about the program 
and believe that youth are benefiting. Three of the CF programs, all four SF programs and 
three of the 10 SWE programs provided program documentation that included testimonials 
and anecdotal information on the benefits to participants. Some examples include: 

• “My internship. . . .was instrumental in getting me a job.” (CF) 

• “..now I feel more confident about entering the workforce” (SF) 

• “The skills I learned will greatly benefit my future employment. I learned many new 
skills while working here.” (SWE) 

Benefits to Employers 

Very little evidence was available to assess the benefits of YES programs to employers. 
Although approximately two-thirds (65%) of the YES departmental programs administer 
post-program questionnaires to employers, few compile or analyze the results. Only 
one program, in the SWE stream, was able to provide a summary of some of the benefits. 
These included that the student provided useful work (31%); reduced the pressure on 
other employees (28%); and enabled an expansion of activity (27%). 

Some of the program documentation provided by program representatives contains 
testimonials and anecdotal information from employers; however, they generally recount 
benefits to youth, rather than to the employers themselves. 
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5. Summary and Conclusions 



5.1 Summary 

The objective of the evaluation was to inform any necessary changes to the Youth 
Employment Strategy (YES) early in the life-cycle of the initiative. The focus of the 
evaluation was on the horizontal aspects of the initiative, in particular on how the 
programs operate collectively to achieve the overall goals of the Strategy. This section 
provides a high level summary of the findings presented in this document and 
conclusions derived from all the research and analysis conducted as part of this study. 

Governance. Selected key elements of the horizontal governance structure are in place, such 
as the basic interdepartmental committees; program policy, design and related operational 
documents (some in draft form); draft data sharing agreements between Human Resources 
and Skills Development Canada (HRSDC) and YES departments and agencies 
(e.g., memoranda of understanding); and basic Participant Information Eorms (PIEs). Initial 
steps toward the development of a computerized PIE and the centralization of the PIE data 
have been undertaken. The departments and agencies have been meeting to discuss and 
resolve horizontal performance measurement issues, mainly in relation to the PIEs.^° 

A number of key elements of the governance structure have yet to be implemented. 
Eor example, there is no structured sharing of information for the purpose of reviewing 
progress in achieving common goals and best practices related to delivery. Agreement 
amongst the departments on common data collection tools has not been achieved (e.g., on 
participant characteristics beyond the short list collected as part of the PIE, and post 
program survey instruments and surveys on outcomes). Neither have the mechanisms 
been developed for tracking youth participants over a number of years to support 
planning activities, to assess participant selections and the nature of interventions 
provided, and to provide evidence on results in the form of annual performance reports 
and summative evaluation studies. 



The YES Interdepartmental Operations Committee is preparing a TBS submission regarding the authorization to collect 
the Social Insurance Number from YES participants. The Committee is also in the process of signing MOUs between 
the lead department and participating departments to authorize the sharing of participant information for purposes of 
horizontal accountability and evaluation. 
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Many of the difficulties associated with horizontal management that YES has 
encountered may be partly attributable to weaknesses in the horizontal design; 
in particular, the value and purpose of horizontality has not been defined in detail. 
This makes it difficult for the Interdepartmental Operations Committee, the lead 
department and other member departments and agencies to establish clearly the purpose 
of their roles, and for participating departments to function effectively as part of what the 
Interdepartmental Operational Guidelines call a “community.” Better definition of 
the expected results of horizontality would assist in: 

• explaining precisely what the lead department is accountable for and the roles of the 
other departments; 

• identifying the purpose of planning and whether there needs to be a more detailed 
master plan for YES as a whole; and 

• identifying the types of decisions the Interdepartmental Operations Committee should 
be making. 

Eor example, the management responsibilities that would appear to be associated with 
leading and planning and with the reporting of results on the realigned YES as a 
horizontal initiative, as outlined in broad terms in the umbrella Results-based 
Management and Accountability Eramework (RMAE), are numerous and considerable. 
They include the administration of formal and informal interdepartmental meetings 
(including bilateral meetings as necessary), preparation of and revisions to program 
documents on program policy, operations and results. They also involve the development 
and updating of skills assessment tools, the review of departmental programs in terms of 
participant selection, interventions, and results measurement, the development and 
administration of common survey instruments, the development and updating of a 
centralized database, the assessment of data integrity, reallocations of investments, and 
follow-up on required changes to any of these. In this regard, the YES management 
documents do not make explicit references regarding how these would be carried out. 
Nor do they present a budgeted line item to highlight costs and human resources required 
to support this work. A YES secretariat to bring attention to and to support the ongoing 
work requirement related to such activities may be warranted. 

Program implementation. The documentation on program parameters is not always 
consistent and the relevance of those described in the Interdepartmental Operational 
Guidehnes is unclear. This comphcated the task of assessing program implementation against 
them. It appears, nonetheless, that in large measure departments have implemented their 
YES programs in accordance with the umbrella terms and conditions (although, a key feature 
of the realigned YES, participant skills assessment, has not been implemented as envisioned). 
There are some programs that do not meet all of the umbrella parameters, but these 
exceptions are covered for the most part in departmental program-specific terms and 
conditions. In some cases, program implementation was still underway in 2004-05. 
While many of the departmental programs had not changed much as a result of the 
realignment, it may be beneficial for YES to take a fresh look at program parameters as 
designed against program parameters as implemented; and highlight and / or correct any 
differences between umbrella and department specific documents. Departments value 
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their own YES programs highly and are primarily focused on their own mandates over 
the broader YES objectives. 

Program delivery models. The YES design allows for different delivery approaches. 
It is part of the strength of the YES that each participating YES department or agency is 
accountable for the management of its own YES program or programs that are closely 
integrated into the particular activities of each participating department. This is how the 
Strategy penetrates different economic sectors and reaches different types of employers, 
thus placing a wide range of work experiences at the disposal of program applicants. 

Benefits to participants. As a result of the different approaches, some of the departmental 
programs are emphasizing interventions whose objectives are broader than “pure skills 
development” and many reported achieving benefits that are not reflected in the umbrella 
RMAE, the departmental RMAEs, or the YES horizontal performance measurement system. 
This is important as the main focus of the performance measurement framework, including 
the summative evaluation, is to provide evidence on program results. The design of the 
summative evaluation can provide an opportunity to review the performance / results 
indicators and the possible effects of the program. In this regard, a review of the 
departmental program theories and a re-examination of the alignment of expected program 
outcomes with these would be useful. This activity could be informed by the Eiterature 
Review that was prepared for the YES in 2004 and the Workshop with experts that was held 
in November 2004. In this way, managers may re-examine the results indicators 
and measures for the individual departmental programs, in the context of the overall spirit 
and objective of the YES. Any changes would have to be integrated into the YES 
performance measurement strategy and associated data collection procedures. 

The Collection and use of Performance Data in Ongoing Performance Management 
Decisions. Most YES programs are collecting basic participant information. About 
two thirds of programs collect data from participants and employers at program exit. 
Eew conduct any follow-up surveys with youth participants; only one conducts follow-up 
surveys with employers. A review of data collection tools against the immediate and 
intermediate outcome indicators in the RMAE found that the programs are not collecting 
all of the data as per the design. No common data collection tools for YES programs 
have been implemented and this has resulted in inconsistencies in data collection across 
programs and in the frequency of collections. With respect to the data that is collected, 
views varied regarding its use for decision making. Of those who said that the data were 
used to make program changes or improvements, few specific examples were provided. 
Some interviewees said that there was a lack of time and / or resources to do any further 
analysis. In cases where interviewees claimed that the data were not being used, several 
also expressed a growing disincentive to continue collecting the information. This has 
implications for evaluating performance at the program and horizontal levels. 

Horizontal Measurement of Results. With respect to the reporting of results achieved 
by the YES, there is a strong emphasis on doing so at the horizontal level. In this respect, 
it is crucial for the YES to have reliable participant / performance information from all 
programs that can be analyzed from the horizontal perspective. This would be facilitated 
by the development of a horizontal program participant database and a coherent and 
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detailed performance measurement and evaluation strategy, including agreement amongst 
partners on core participant outcomes to be tracked. 



5.2 Conclusions 

The realigned YES has been implemented as a horizontal initiative. Basic committee 
structures, documents (some remain in draft form), and management processes aimed at 
guiding and reporting on program operations and results are evident. Member departments 
and agencies have demonstrated the willingness to work together to discuss and resolve 
common issues and continue to do so. For the most part, YES departmental programs have 
introduced the collection of basic personal information for purposes of accountability and 
evaluation. Many departmental programs collect additional performance information. (Data 
integrity has not been verified.) 

However, as indicated by the findings summarized in this study, mid-course corrections 
may be warranted to each of these basic components. In particular, the YES would 
benefit from: 

• Clarification of the roles and responsibilities for the Interdepartmental Operations 
Committee and its general membership, and more specifically those for its lead department 
and TBS, and the roles and responsibilities for the two sub-committees. 

• Descriptions of the management and performance measurement activities that the 
Interdepartmental Operations Committee and its members are to undertake in order to 
guide, assess, and report on YES investments. For example, they could undertake: 

A re-examination of departmental program theories, expected outcomes, 
participant selection criteria as applied, interventions provided, and performance 
indicators and methods of measuring results at both the horizontal and 
departmental program levels. YES may benefit from describing in more detail 
how departmental programs could measure and document skills improvements 
and the part their interventions play in assisting youth on the path to “sustained 
employment”; 

A review of the departmental programs by program managers against both 
umbrella and department- specific Ts and Cs; and clarifications regarding which 
Ts and Cs take precedence; and 

Full implementation of YES horizontal performance measurement tools, allowing 
ongoing assessment of data integrity and enhanced annual horizontal reporting 
(e.g., on participant characteristics, average investments, immediate outcomes by 
horizontal and departmental program). 

• Elucidation of the resources required to fulfill the ongoing horizontal management 
responsibilities and the structure and processes that the Interdepartmental Operations 
Committee will use to ensure that these are fulfilled. 
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Overall, revisions to YES management and performance measurement strategy could be 
summarized in related management and operational documents. This would also be a 
means of reporting on mid-course corrections that are being implemented in response to 
this formative evaluation and as a result of ongoing management assessment. 
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Annex A: 

Supporting Technical Documents 



YES Evaluation Findings - Technical Reports 

• Formative Evaluation of the Youth Employment Strategy (YES), prepared for Human 
Resources and Skills Development Canada by Consulting and Audit Canada, 
March 2005. 

• Executive Overview of Eindings from the Eormative Evaluation of YES, Presentation 
to the YES Evaluation Steering Committee, Eebruary 22, 2005, presented by CAC. 

• Presentation: CAC [Consulting and Audit Canada] Draft bindings from the 
Eormative Evaluation of YES, Eor Discussion with the YES Evaluation Working 
Group, January 14, 2005. 

Departmental Program Summary Reports 

• Summary of Eindings: YES Eormative Evaluation for Career Eocus Program, 
Agriculture and Agri-Eood Canada, prepared by Consulting and Audit Canada, 
March 2005. 

• Summary of bindings: YES Eormative Evaluation for Science Horizons Canadian food 
Inspection Agency, prepared by Consulting and Audit Canada, March 2005. 

• Summary of Eindings: YES Eormative Evaluation for International Youth Internship 
Program, Canadian International Development Agency, prepared by Consulting and 
Audit Canada, March 2005. 

• Summary of bindings: YES Eormative Evaluation for Housing Internship Program, 
Canada Mortgage and Housing Corporation, prepared by Consulting and Audit 
Canada, March 2005. 

• Summary of bindings: YES Eormative Evaluation for Young Professionals 
International, Eoreign Affairs Canada, prepared by Consulting and Audit Canada, 
March 2005. 

• Summary of bindings: YES Eormative Evaluation for Science and Technology Youth 
Internships Program, Eisheries and Oceans Canada, prepared by Consulting and Audit 
Canada, March 2005. 

• Summary of Eindings: YES Eormative Evaluation for International Environmental 
Youth Corp, Environment Canada, prepared by Consulting and Audit Canada, 
March 2005. 
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• Summary of Findings: YES Formative Evaluation for Science Horizons, Environment 
Canada, prepared by Consulting and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Career Focus, Human Resources 
and Skills Development Canada, prepared by Consulting and Audit Canada, 
March 2005. 

• Summary of Findings: YES Formative Evaluation for International Academic 
Mobility, Human Resources and Skills Development Canada, prepared by Consulting 
and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Summer Career Placements, 
Human Resources and Skills Development Canada, prepared by Consulting and Audit 
Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Sectoral Career Focus, Human 
Resources and Skills Development Canada, prepared by Consulting and Audit Canada, 
March 2005. 

• Summary of Findings: YES Formative Evaluation for Community Access Program - 
Youth Initiative, Industry Canada, prepared by Consulting and Audit Canada, 
March 2005. 

• Summary of Findings: YES Formative Evaluation for NetCorps Canada International, 
Industry Canada, prepared by Consulting and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Student Connections, Industry 
Canada, prepared by Consulting and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for SchoolNet, Industry Canada, 
prepared by Consulting and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Indian and Northern Affairs, 
prepared by Consulting and Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Internship Program with 
Innovative SMEs and Collaborative Research Internships Program Industrial Research 
Assistance Program, National Research Council, prepared by Consulting and Audit 
Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Science and Technology 
Internships Program, Natural Resources Canada, prepared by Consulting and Audit 
Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Young Canada Works in 
National Parks and National Historic Sites, Parks Canada, prepared by Consulting and 
Audit Canada, March 2005. 

• Summary of Findings: YES Formative Evaluation for Young Canada Works, Canadian 
Heritage, prepared by Consulting and Audit Canada, March 2005. 
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Background Documents 



• Evaluation of Youth Employment Programs - Executive Overview” by W. Craig Riddell, 
Department of Economics, University of British Columbia and Arthur Sweetman, 
School of Policy Studies, Queen's University, Eebruary 22, 2005. 

• Evaluation of Youth Employment Programs” by W. Craig Riddell, Department of 
Economics, University of British Columbia and Arthur Sweetman, School of Policy 
Studies, Queen's University, Eebruary 22, 2005. 

• The Youth Employment Strategy, Key Messages from the Panel of Expert Workshop, 
held on November 23, 2004, at the Sheraton Ottawa, 150 Albert Street, Ottawa, Ontario. 

• “Governance, Accountability and Horizontal Operations, presentation to the YES 
Workshop [on design of formative evaluation], June 22, 2004, prepared by Consulting 
and Audit Canada. 
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